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The Glauber dynamics of the Hopfield model at low storage level is considered. We analytically 
derive the spectrum of relaxation times for large system sizes. The longest time scales are gathered 
in families, each family being in one to one correspondence with a stationary (not necessarily stable) 
point of the static mean-field free-energy. Inside a family, the time scales are given by the reciprocals 
(of the absolute values) of the eigenvalues of the free-energy Hessian matrix. 



Over the past few years, there has been a renewal of interest for the long time dynamics of spin-glass models, which 
may already exhibit non trivial features, e.g. violation of fluctuation-dissipation relations, aging phenomena, ... at 
the mean-field level [1]. In this context, a natural quantity of interest is the spectrum of relaxation times occuring in 
the Glauber dynamics of disordered Ising spin models [2] . Unfortunately, numerical investigations have been so far 
limited to very small sizes [4]. Moreover, to the best of our knowledge, no analytical studies of the relaxation times 
spectra have been performed yet due to the technical difficulties arising in the diagonalization of the Glauber matrix. 

In this letter, we focus on the Glauber dynamics of the Hopfield model at low storage [6]. This system is simple 
enough to be analytically solved along the lines of [5]. Our main result is that the longest time scales are gathered 
in families, each family being in one to one correspondence with a stationary (not necessarily stable) point of the 
static mean-field free-energy. Inside a family, the time scales are given by the reciprocals of the absolute values of the 
eigenvalues of the free-energy Hessian matrix. As a consequence, our study strcnghtens the close relationship between 
long times dynamics and static properties in disordered mean-field models [1]. 

We consider a Hopfield model including N Ising spins Si, i — 1, . . . , N and p quenched patterns £f, fi = 1, . . . ,p. 
In addition to its intrinsic interest as a paradigm for attractor neural network [7] , the Hopfield model may be seen as 
a spin glass system smoothly interpolating between the Mattis model [8] (when p — 1) and the much more involved 
Sherrington-Kirpatrick model (for infinite p) [3]. Its Hamiltonian depends on the spin configuration S = [Si, ■ ■ ■ , Sjv) 

1 N 

through the set of p overlaps rn M (S) = — Si£f and reads 
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The thermodynamics of the above Hamiltonian has been studied in [7]. At low storage level, that is when p keeps 
finite as the size N is sent to infinity, the free-energy density is particularly simple, 
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where f3 is the inverse temperature and <C ^= ^ F({£i }) denotes the site average. In the above expres- 
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sion (2), the overlaps are the solutions of the saddle-point equations 
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with the smallest free-energy. At high temperature, the system lies in the paramagnetic phase m M = 0, V/x. Below the 
critical temperature 1//3 C = 1 , there appear many other solutions with non zero overlaps. The low temperature phase 
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is characterized by a condensation along some particular patterns, while spurious mixtures give rise to metastable 
states. 

These equilibrium states may be seen as the attractors of a retrieval dynamics [2] starting from a given initial 
configuration (which is often but not necessarily chosen as a corrugated version of the quenched patterns). One 
possible evolution scheme is provided by the usual Glauber dynamics : given a configuration S, the spin at site i is 
nipped with probability rate w(S, i) = 5(1 — Si tanh[/3/ij(S)]) and left unchanged with rate 1 — w. The instantaneous 
magnetic field hi reads from (1), 



The probability distribution P(S,t) satisfies the master equation 

N 

P(S t) = 

dt 



J J>(S,t) = £(to(S',i)-P(S',t) - »(S,i)P(S,t)) , (5) 
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where the spin configuration S l is obtained from S by flipping spin i. Due to its linear structure, eqn. (5) may be 
recast in a more suitable operator formalism. To each configuration S is associated a vector |S), which form the 

basis of a 2 w -dimensional linear space V. The probability distribution then becomes a vector | P(t) ) = P (S, t) | S). 

s 

We now define the spin flip and magnetization operators, respectively denoted by of and of through of | S ) = | S* ) 
and of I S) = Si I S). ft is easy to check that the vectorial operator \cr — \{a x ,a v = icr x a z ,a z ) satisfies the usual 
commutation relations for a quantum spin 1/2. The master equation (5) now acquires the following compact form 



, 1 N 

-\P(t))=W\P(t)) , W=-E(af-l)(l-<7ftanh(/?/ l ,)) , (6) 
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where the local field operator hi is obtained by replacing in (4) the spins Si with their corresponding magnetization 
operators of. Due to the detailed balance conditions, the operator W s = cxp((3H/2)W exp(— (3H/2) is Hermitian and 
reads 
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of (cosh(/3/i i ))^ 1 + of tanh(/?/K) 
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The spectrum of W s , which is the same as Ws one, we shall now analyze. From a rigorous point of view, all eigenvalues 
are strictly negative except the zero mode corresponding to the equilibrium Gibbs measure. First, as far as long time 
dynamics is concerned, all large (in absolute value) eigenvalues, that are associated to fast decaying modes, may be 
discard. Secondly, for very large system sizes (N » 1), indeed the limit of physical interest, the zero eigenvalue of W s 
is expected to become almost degenerate when (3 > (i c . Dynamical processes taking place over exponentially large 
times, or more generally times diverging with N will not be considered. We therefore focus only on long but finite 
time scales. 

To do so, we shall perform an expansion of W s in powers of l/\/N and keep only the non vanishing terms in 
the thcrmodynamical limit. This expansion is made possible by the mean-field nature of the Hopficld Hamiltonian 
(1). For a given configuration S, the dominant contributions Hi = ^""^ ^m^(S) to the local fields hi (4) take only 

n 

2 P different values when scanning all sites i. Let us partition the N sites in 2 P blocks GV where r is a p binary 
component vector; site i belongs to the only group GV such that Hi equals Hy = r AI m M (S) [9]. We restrict to 

quenched patterns that differ from each other on a macroscopic number of sites, so each block G T contains a number 
Nc T of sites of the order of TV; note that if the patterns are randomly drawn from an unbiased distribution c T = 2~ p 
in the limit of large sizes. 

After some simple algebra, the evolution operator may be rewritten as W s — ^ W-f with 

T 

W* T = -^c T + r T (cosher))- 1 + J Z T tanh(/3if T ) - ^c T (l - tanh(/3ff x ) 2 ) + O (jj^ (8) 
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where the angular momenta and the field operators respectively read 



J T = ij2^ and H T = ^Y.il T ^ J h ■ ( 9 ) 

ieG T r) |U=i 

The last two terms on the r.h.s of (8) stem from the contributions pSi/N to the local fields (4). The angular momenta 
J T and J T i commute for any two different groups r ^ r' and so does any J T with VF S . Thus, the angular momenta 
may be used as quantum numbers labelling the different invariant subspaces. 

It can be checked that the subspace V s corresponding to the maximum value of each angular momentum J T = \Nct 
contains the totally symmetric vectors | {M T } ) having magnetizations Mj- = —Ncr, —Nc T + 2, . . . , Nct in blocks 
G T - Indeed, the vector | {M T — Nc T }) = \ +,+,... , + ) belongs to V s since its angular momentum component along 
the z axis equals N/2 and is maximal. Moreover, the lowering operators leave V s invariant and | {Nc T — 2d T }) = 
®(J^) d T | {Nct})- In addition, the equilibrium Gibbs vector belongs to V s . We shall see in the following that not 
only the Gibbs mode but also all the eigenvectors corresponding to the smallest eigenvalues lie in V s . 

At infinite temperature, the system is purely diffusive and the equilibrium state | ) equals the normalized sum 
of all | S)s. The average values of the angular momentum read (0|J a |0) = (0|J Z |0) = , (0|J X |0) = N/2. 
For low-lying excitations that is plane waves on the A^-dimensional hypercube, the mean value of the angular 
momentum (to | J \ lo) equals the one in the equilibrium state up to non extensive terms. The same picture holds at 
finite temperature, but not at phase transition boundaries. More precisely, equilibrium or quasi-stationary modes 
|0) are expected to correspond to well-defined direction of the angular momenta Jr in the three-dimensional space. 
This direction smoothly varies when exploring the excited modes slightly above | ) 1 . To capture the spectrum of 
low-lying states, we therefore proceed in two steps : first identify the preferred directions of the angular momentum 
corresponding to quasi-cquilibrium modes and then compute the fluctuations around these positions. 

This approach, reminiscent of semi-classical approximations and spin wave theory [11], may be made more precise in 
the following way. Consider the Holstein-Primakoff (HP) representation of the angular momentum vectorial operator 
[10] in terms of Bose operators 

L*- = ^Nc T - a\-a T 
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which is valid for fixed angular momentum \Nct and as long as the number of quanta does not exceed Nc T . The 

Bose operators fulfill the usual commutation relation [a T ,a^-,] = 8 T T i. The large N expansion will be consistent 
provided that we enforce the number of quanta to keep finite. To do so, we align the HP representations (10) along 
the quasi-equilibrium directions of the angular momenta 

J£ = cos T L X T - sin B T L Z T , J T = L y T , J T = sin 6 T L X T + cos B T L Z T (11) 
in every blocks. We may now expand the evolution operator W s = N.Wq + VN .W( + W 2 S + 0(1/ VN) and find 



W? = - V^, (cosfl T sinh^r - sin T ) x 
2 cosh K T 

4 + or - E E ^ + ar]) 1 (12) 



1 Activated processes between wells can not be obtained this way but they correspond to infinite time scales we do not 
investigate here. 
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where 



(13) 



Both Wq and W( vanish when the angles are such as sin #-7- = tanh K T for all blocks r. Expression (13) implies 
that the 2 P angles may be computed through the knowledge of the p parameters to m = cq sin 8-q rf . It turns out 

V 

that the resulting self-consistent equations for the m M s are identical to the equilibrium saddle-point conditions (3), as 
could be expected from the above discussion. 

As a consequence, the smallest eigenvalues gather together in different families T, in one to one correspondence 
with the stationary points of the free energy (2). In each family T, we are left with the spin- wave part of the evolution 
operator 



W S 



\ [(or + 4)(0 2 - 1)t,t'(ot' + 4')] - E [4«r + ^(1 - O t ,t) 
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where the 2 P x 2 P matrix O reads 



T.T 



= 1 



T.T' 
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At least 2 P — p eigenvalues of O equal one. A straigthforward calculation show that the difference between the traces 
of the n th powers of O and of the free-energy Hessian matrix, see (2, 3), 
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equals 2 P — p for all integers n. In the above expression, the Hessian depends on the particular family T i.e. saddle- 
point under consideration. Therefore, the a priori non unit eigenvalues of O, hereafter called AjS, j — 1, . . . ,p are 
precisely the eigenvalues of the second derivatives matrix of the static free-energy. 

To achieve the calculation of the spectrum of relaxation times, we now perform an orthogonal transformation over 
the Bose operators to diagonalize O and obtain 
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The first term on the r.h.s. of (17) includes p interacting bosons which can be decoupled by means of a Bogoliubov 
transformation 



cosh ( 
sinh< 



sinh( 
cosh< 
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where <j>j 



I In 



Xj | . We can finally rewrite in a completely diagonal form, 



Wi = -j2\\ j \(c]c j +A j ) 
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E 
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where Aj = 1 if Xj < and Aj = if Xj > 0. In this form, the slow eigenmodes of the Glauber matrix arc 
seen to correspond to the eigenvectors of a quantum harmonic oscillator and can be easily calculated [5] . Note that 
transformation (18) requires that no eigenvalue vanishes. Indeed, for a marginal saddle-point !F, the large N expansion 
has to take into account that the number of quanta a 1 a becomes of the order of v^V- 

We can follow the same procedure to diagonalize W s in the subspace labelled by angular momenta { J r = \Nc-i — 
j T } where all jV s are finite; we obtain for W^- expression (19) plus the constant term — j T . Therefore, the smallest 
eigenvalues do not belong to such subspaces. If the jrs are not finite as N goes to infinity, the diverging evolution 
operators Wq and W( can never be simultaneously cancelled through adequate rotations. 
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Starting from a given initial condition, the evolution in the parameter space {m M } will be deterministic [12] and 
follow the macroscopic law 



= ' (20) 

giving rise to a gradient descent of the free-energy landscape down to some local minimum. When the number p 
of patterns increases, this free-energy landscape becomes very complex, e.g. the number of stationary points scales 
exponentially with p [7] . Each fixed point has a basin of attraction; a point for a local maximum, a line for points with 
only one stable direction, . . . , a manifold of dimension N — I for a stationary point of index /. However, because of 
pure entropic effects, some stationary points have much wider basins of attractions that other ones, that is the number 
of microscopic configurations associated to them may be extremely large. For instance, a random and unbiased initial 
condition would fall into the basin of the paramagnetic saddle-point m M = 0, V/i. 

Generally speaking, when the starting point lies in a basin of attraction of a stationary point {m^}f of the 
free-energy, the system relaxes toward this stationary point. Later on, the order parameters keep stucked to their 
saddle-point values for all but finite times, even if T is unstable from a thermodynamical standpoint. This description 
is reminiscent of the scenario proposed in [13]. The time scales we have calculated here govern the \j\f~N fluctuations 
around the stationary point. Fluctuations in the stable directions, that are related to positive AjS, decay with 
relaxation times equal to 1/Xj and eventually equilibrate (Aj = 0). As for fluctuations along the unstable directions 
associated to negative XjS, they exponentially grow (Aj = 1) with some characteristic 'escape' times l/|Aj|. In both 
cases, the number of affected spins is of the order of ^J~N and the intensive quantities remain unchanged. 
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